Driving support device, driving support system, and driving support method

ABSTRACT

In a driving support device, an image output unit outputs image information including a vehicle object representing a vehicle and a sign object representing a sign, to a display unit. An operation signal input unit receives an operation signal of a user for moving, in the image displayed on the display unit, the vehicle object to the position of the sign object or for moving the sign object to the position of the vehicle object. A command output unit outputs a command corresponding to the display content in the sign object when the operation signal is received, to an automatic driving control unit that controls automatic driving.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. national stage application of the PCTInternational Application No. PCT/JP2016/003383 filed on Jul. 19, 2016,which claims the benefit of foreign priority of Japanese patentapplication No, 2015-152851 filed on Jul. 31, 2015, the contents all ofwhich are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a technique that supports a drivingoperation instruction of a driver to a vehicle during automatic driving.

BACKGROUND ART

In recent years, developments in automatic driving have progressed. Forautomatic driving, autonomy levels defined in 2013 by the NationalHighway Traffic Safety Administration (NHTSA) are classified as noautonomy (level 0), specific-function autonomy (level 1),complex-function autonomy (level 2), semi-automatic driving (level 3),and full-automatic driving (level 4). Level 1 is a driving supportsystem that automatically performs one of acceleration, deceleration,and steering, and level 2 is a driving support system that automaticallyperforms two or more of acceleration, deceleration, and steering incoordination. In all cases, a driver is required to be involved in adriving operation. Autonomy level 4 is a fully-automatic driving systemthat automatically performs all of acceleration, deceleration, andsteering, and thus a driver is not involved in a driving operation.Autonomy level 3 is a semi-fully-automatic driving system thatautomatically performs all of acceleration, deceleration, and steering,but if necessary, a driving operation is performed by a driver.

As a form of automatic driving, a form in which a driver does notoperate an existing driving operation unit such as a steering, anaccelerator pedal, or the like but instructs a specific drivingoperation such as traffic lane change, passing, following traveling, orthe like to a vehicle by issuing a command to a vehicle is considered.In this form, a user interface in which there are fewer erroneousoperations is required. A technique that extracts a guide sign from theoriginal image and selectively displays the guide sign based on atraveling direction or the like is known (for example, refer to PTL 1).

CITATION LIST Patent Literature

PTL 1: Japanese Patent Unexamined Publication No. 2014-074627

SUMMARY OF THE INVENTION

In order to solve the above-described problems, a driving support deviceaccording to an aspect of the present disclosure includes an imageoutput unit that outputs image information including a vehicle objectrepresenting a vehicle and a sign object representing sign information,to a display unit. The driving support device further includes anoperation signal input unit that receives an operation signal of a userfor moving, in the image displayed on the display unit, the vehicleobject to the position of the sign object or for moving the sign objectto the position of the vehicle object. The driving support devicefurther includes a command output unit that outputs an automatic drivingcontrol command corresponding to the display content in the sign objectwhen the operation signal is received, to an automatic driving controlunit that controls automatic driving.

As another aspect of the present invention, an aspect in which anexpression of the present invention is converted into a device, asystem, a method, a program, a recording medium in which a program isrecorded, and an automatic drive vehicle equipped therewith byarbitrarily combining the above components is also effective.

According to the present disclosure, a driver can intuitively andconveniently instruct a specific driving operation to a vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a vehicleaccording to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a basic sequence ofdetection unit, automatic driving controller, HMI controller, displayunit, and input unit in FIG. 1.

FIG. 3 is a diagram illustrating an example of a basic flowchart forexplaining processing of HMI controller in FIG. 1.

FIG. 4 is a flowchart for explaining update processing of a receptionmode.

FIG. 5 is a flowchart illustrating an example of determinationprocessing in a case where a gesture operation for instructing aspecific operation using a road guide sign is input in step S9 of FIG.3.

FIG. 6 is a flowchart illustrating a processing example of issuing acommand corresponding to the content of the road guide sign by a gestureoperation.

FIG. 7A and FIG. 7B are diagrams illustrating an example of issuing acourse change instruction command or a route change instruction commandusing a destination guide sign at an intersection.

FIG. 8A and FIG. 8B are diagrams illustrating an example of issuing acourse change instruction command or a route change instruction commandusing a traffic lane guide sign.

FIG. 9A and FIG. 9B are diagrams illustrating an example of issuing asetting instruction command for setting a passing spot using an exitguide sign of a toll road.

FIG. 10A and FIG. 10B are diagrams illustrating another example ofissuing a setting instruction command for setting a passing spot usingan exit guide sign of a toll road.

FIG. 11A and FIG. 11B are diagrams illustrating an example of issuing adestination setting instruction command for setting a destination usinga parking lot guide sign.

FIG. 12A to FIG. 12C are diagrams illustrating another example ofissuing a course change instruction command using a destination guidesign at an intersection.

FIG. 13A and FIG. 13B are diagrams illustrating an example of issuing atraffic lane change instruction command and a course change instructioncommand using a guide sign on a traffic lane.

DESCRIPTION OF EMBODIMENTS

Prior to the description of an exemplary embodiment of the presentinvention, problems in the related art are briefly described.

In a case where automatic driving is applied to a vehicle, it takes sometime for a driver to confirm a sign and then instruct a course usingseparate means (a steering, an accelerator, or the like) located atanother location. In a design in which each switch is designed for aspecific driving operation such as traffic lane change, passing,following traveling, or the like, in a case where a driver operates theswitch without looking at a his/her hand while checking a peripheralsituation of the vehicle, there is a possibility of an erroneousoperation. Further, the correspondence between a switch operation andautomatic traveling control is not intuitive, and selecting aninstruction among many switches or selecting a changed function withreduced switches according to a situation is a complex task. There is aproblem in that a wide range of users including a person who has notbeen driving up to now, a person who wants to continue to drive eventhough the driving capability of the person has been reduced, or thelike, cannot use an automatic drive vehicle without training.

The present disclosure has been made in consideration of such asituation, and an object thereof is to provide a technique that allows adriver to intuitively and conveniently instruct a specific drivingoperation to a vehicle.

Hereinafter, embodiments of the present invention will be described withreference to the drawings. FIG. 1 is a block diagram illustrating aconfiguration of vehicle 1 according to an embodiment of the presentinvention, and illustrates a configuration related to automatic driving.Vehicle 1 (an own vehicle) with an automatic driving mode includesdriving support device (human-machine interface (HMI) controller) 10,automatic driving control device (automatic driving controller) 20,display device 30, detection unit 40, and driving operation unit 50.

Display device 30 includes display unit 31 and input unit 32. Displaydevice 30 may be a head unit such as a car navigation system, a displayaudio, or the like, a portable terminal device such as a smart phone, atablet, or the like, or a dedicated console terminal device.

Display unit 31 is a liquid crystal display, an organicelectro-luminescence (EL) display, or a heads-up display (HUD). Inputunit 32 is a user interface that receives an input of a user. Displayunit 31 and input unit 32 may be an integrated touch panel display. Suchas a proximity touch panel that can detect proximity of a hand on atouch panel or a touch pad or a position of a finger due to a hoveroperation, one that receives a gesture input at a position apart from apredetermined distance from the display unit may be used. Input unit 32may have an input device such as a mouse, a stylus pen, a trackball, orthe like for assisting the gesture input. A pen that emits visible lightor infrared light may also be used.

Display unit 31 and input unit 32 may be physically separated from eachother, instead of being an integrated touch panel display. For example,input unit 32 includes a sensor such as a camera or the like, and may bea non-contact type input device that allows a gesture operation input inthe air. For example, an operation method that starts to drag with agesture by pointing a target with a finger and making a thumb finger andan index finger close and shut together, and ends to drag with a gestureby making a thumb finger and an index finger separated from each other,is considered.

Driving support device 10 and display device 30 may be connected to eachother by wire communication such as a dedicated line, a controller areanetwork (CAN), or the like, or may be connected to each other by wirecommunication or wireless communication such as USB, Ethernet(registered trademark), Wi-Fi (registered trademark), Bluetooth(registered trademark), or the like.

Detection unit 40 includes position information acquisition unit 41,sensor 42, speed information acquisition unit 43, and map informationacquisition unit 44. Position information acquisition unit 41 acquires acurrent position of vehicle 1 from a global positioning system (GPS)receiver. Sensor 42 is a general term for various sensors for detectinga situation outside the vehicle and a state of vehicle 1. As a sensorfor detecting a situation outside the vehicle, for example, a camera, amillimeter wave radar, a light detection and ranging (laser imagingdetection and ranging, LIDAR), a temperature sensor, a pressure sensor,a humidity sensor, an illumination sensor, or the like is mounted. As asituation outside the vehicle, a road situation in which the own vehicletravels or an environmental situation including weather, and an ownvehicle peripheral situation such as a traveling position or a travelingstate of an another vehicle traveling at the periphery of the ownvehicle are considered. Any information outside the vehicle that can bedetected by the sensor may be considered. As a sensor for detecting astate of vehicle 1, for example, an acceleration sensor, a gyro sensor,a geomagnetic sensor, an inclination sensor, or the like is mounted.Speed information acquisition unit 43 acquires the current speed ofvehicle 1 from a vehicle speed sensor. Map information acquiring unit 44acquires map information around the current position of vehicle 1 from amap database. Map database may be recorded on a recording medium invehicle 1, or may be downloaded from a map server via a network in use.

Detection unit 40 and automatic driving control device 20 are connectedto each other by wire communication such as a dedicated line, a USB, anEthernet (registered trademark), a controller area network (CAN), or thelike. A configuration in which data acquired and detected by detectionunit 40 is directly output from detection unit 40 to driving supportdevice 10 may be employed.

Driving operation unit 50 includes steering 51, brake pedal 52,accelerator pedal 53, and blinker switch 54. In an automatic drivingmode according to the present embodiment, acceleration, deceleration,steering, and blinker blink are a target of automatic control byautomatic driving control device 20. In FIG. 1, an operation unit whenthese controls are performed manually is drawn. Information indicatingthat driving operation unit 50 is slightly moved by a driver in a manualway may output to driving support device 10.

Steering 51 is an operation unit for steering the vehicle. When steering51 is rotated by a driver, the traveling direction of the vehicle iscontrolled through a steering actuator. The steering actuator can beelectronically controlled by a steering electronic control unit (ECU).

Brake pedal 52 is an operation unit for decelerating vehicle 1. Whenbrake pedal 52 is depressed by a driver, the vehicle is decelerated viaa brake actuator. The brake actuator can be electronically controlled bya brake ECU.

Accelerator pedal 53 is an operation unit for accelerating vehicle 1.When accelerator pedal 53 is depressed by a driver, an engine rotationspeed and/or a motor rotation speed is controlled via an acceleratoractuator. In a pure gas car, the engine rotation speed is controlled. Ina pure electric vehicle, the motor rotation speed is controlled. In ahybrid car, both of the engine rotation speed and the motor rotationspeed are controlled. The accelerator actuator can be electronicallycontrolled by at least one of an engine ECU and a motor ECU.

Blinker switch 54 is an operation unit for blinking a blinker so as tonotify a course of the vehicle to the outside. When blinker switch 54 isturned on/off by a driver, the blinker is turned on/off via a blinkercontroller. The blinker controller includes a drive circuit such as arelay controlling power supply to a blinker lamp, or the like.

Each of the steering ECU, the brake ECU, the engine ECU, the motor ECU,and the blinker controller and automatic driving control device 20 areconnected to each other by wired communication such as a CAN, adedicated line, or the like. Each of the steering ECU, the brake ECU,the engine ECU, the motor ECU, and the blinker controller respectivelytransmits a state signal indicating the state of each of a steering, abrake, an engine, a motor, and a blinker lamp to automatic drivingcontrol device 20.

In the automatic driving mode, each of the steering ECU, the brake ECU,the engine ECU, and the motor ECU drives the corresponding actuatoraccording to a control signal supplied from automatic driving controldevice 20. In a manual driving mode, a configuration in which aninstruction is directly transferred from each of steering 51, brakepedal 52, and accelerator pedal 53 to the corresponding actuator in amechanical way may be employed, or a configuration in which electroniccontrol intervenes via the corresponding ECU may be employed. Theblinker controller turns on/off the blinker lamp according to thecontrol signal supplied from automatic driving control device 20 or aninstruction signal from blinker switch 54.

Automatic driving control device 20 is an automatic driving controllerthat has an automatic driving control function, and includes controlunit 21, memory unit 22, and input-output unit 23. The configuration ofcontrol unit 21 may be realized by cooperation between hardwareresources and software resources or only hardware resources. As thehardware resources, a processor, a read only memory (ROM), a randomaccess memory (RAM), and other large scale integrated (LSI) circuits canbe used, and as software resources, an operating system, an application,and a program such as a firmware or the like can be used. Memory unit 22includes a non-volatile recording medium such as a flash memory or thelike. Input-output unit 23 performs various communication controlaccording to various communication formats.

Control unit 21 calculates a control value for controlling an automaticcontrol target such as a traveling direction of vehicle 1 by applyingvarious parameter values collected from detection unit 40 and variousECUs to an automatic driving algorithm. Control unit 21 transfers thecalculated control value to the ECU or controller of each controltarget. In the present embodiment, the control value is transferred tothe steering ECU, the brake ECU, the engine ECU, and the blinkercontroller. In a case of an electric vehicle or a hybrid car, thecontrol value is transferred to the motor ECU instead of the engine ECUor in addition to the engine ECU.

Driving support device 10 is a human machine interface (HMI) controllerfor performing an interface function between vehicle 1 and a driver, andincludes determination unit 11, image generation unit 12, instructionunit 13, and input-output unit 14. Determination unit 11, imagegeneration unit 12, and instruction unit 13 can be realized bycooperation between hardware resources and software resources, or onlyhardware resources. As the hardware resources, a processor, a ROM, aRAM, and other LSI circuits can be used, and as software resources, anoperating system, an application, and a program such as a firmware orthe like can be used. Input-output unit 14 performs variouscommunication control according to various communication formats.Input-output unit 14 includes image output unit 14 a, operation signalinput unit 14 b, command output unit 14 c, and vehicle information inputunit 14 d. Image output unit 14 a outputs an image generated by imagegeneration unit 12 to display unit 31. Operation signal input unit 14 breceives an operation signal that is input from input unit 32 by anoperation of a driver, a passenger, or a user outside the vehicle, andoutputs the operation signal to determination unit 11. Command outputunit 14 c outputs the command instructed by instruction unit 13 toautomatic driving controller 20. Vehicle information input unit 14 dreceives detection data acquired by detection unit 40 or vehicleinformation generated by automatic driving controller 20, and outputsthe detection data or the vehicle information to image generation unit12.

Automatic driving controller 20 and HMI controller 10 are directlyconnected to each other by a signal line. A configuration in whichautomatic driving controller 20 and HMI controller 10 are connected toeach other via a CAN may be employed. A configuration in which automaticdriving controller 20 and HMI controller 10 are integrated into onecontroller may be also employed.

FIG. 2 is a diagram illustrating an example of a basic sequence ofdetection unit 40, automatic driving controller 20, HMI controller 10,display unit 31, and input unit 32 in FIG. 1. Detection unit 40 detectsown vehicle position information, own vehicle traveling roadinformation, and own vehicle peripheral information, and outputs thedetected information to automatic driving controller 20 (P1). Automaticdriving controller 20 outputs the own vehicle position informationacquired from detection unit 40, the own vehicle traveling roadinformation, and the own vehicle peripheral information to HMIcontroller 10 (P2). HMI controller 10 generates a schematic diagramincluding the own vehicle and the own vehicle peripheral situation basedon the information acquired from automatic driving controller 20 (P3).HMI controller 10 outputs the generated schematic diagram to displaydevice 30, and the schematic diagram is displayed on display unit 31(P4).

A user who views the schematic diagram displayed on display unit 31comes into contact with input unit 32 (P5). Display unit 31 outputscoordinate data at a position at which the contact is detected to HMIcontroller 10 (P6). HMI controller 10 determines a type of the commandbased on the coordinate data acquired from display device 30 (P7). HMIcontroller 10 receives an additional input until a certain period oftime has elapsed (P8 to P12). After determination of the command, HMIcontroller 10 regenerates a schematic diagram indicating that thecommand is being instructed (P8). HMI controller 10 outputs theregenerated schematic diagram to display device 30, and the regeneratedschematic diagram is displayed on display unit 31 (P9). In a case wherethe command corresponding to the gesture operation by the touch of auser does not exist, HMI controller 10 generates a schematic diagramincluding an error message, and outputs the schematic diagram to displaydevice 30, and the schematic diagram is displayed on display unit 31.

When a user who views the schematic diagram indicating that the commandis being instructed comes into contact with input unit 32 (P10), displayunit 31 outputs the coordinate data at a position at which the contactis detected to HMI controller 10 (P11). HMI controller 10 performsadditional command processing based on the coordinate data acquired fromdisplay device 30 (P12). In a case where there is no input of a newcommand in the additional command processing (P12), HMI controller 10issues a command determined in P7 to automatic driving controller 20(P13 and P14). In a case where a new command is input in the additionalcommand processing (P12), HMI controller 10 issues a new command toautomatic driving controller 20. In a case where the new command that isinput is a cancel command, HMI controller 10 cancels the issue of acommand. Processing of overwriting and canceling of the original commanddue to a new command may be performed by automatic driving controller20. In this case, after the command determination processing in P7 andP12, HMI controller 10 transmits the command to automatic drivingcontroller 20, and performs processing of overwriting and cancelingaccording to an internal state of automatic driving controller 20.

Detection unit 40 periodically detects the own vehicle positioninformation, the own vehicle traveling road information, and the ownvehicle peripheral information, and outputs the information to automaticdriving controller 20 (P15). Automatic driving controller 20 determineswhether or not control instructed by the command issued from HMIcontroller 10 is executable (P16), based on the information. In a casewhere it is determined that the control is executable, automatic drivingcontroller 20 outputs a control start notification to HMI controller 10(P17). When the control start notification is received, HMI controller10 regenerates a schematic diagram including a message indicating thatthe control is being performed (P18). HMI controller 10 outputs theregenerated schematic diagram to display device 30, and the regeneratedschematic diagram is displayed on the display unit 31 (P19). Althoughnot illustrated, automatic driving controller 20 calculates a controlvalue for controlling driving operation unit 50 that performs the issuedcommand by applying various parameter values collected from detectionunit 40 or various ECUs to an automatic driving algorithm, and transfersthe control value to the ECU or controller of each control target. Thecontrol value is a specific value for automatically controlling controltarget such as a traveling direction of vehicle 1 or the like. Drivingoperation unit 50 operates based on the specific control value. When apredetermined control value or the detection data acquired by detectionunit 40 is a predetermined value (in a predetermined range), and whenautomatic driving controller 20 determines that a condition of theissued command is satisfied, driving operation unit 50 determines thatperforming of the command is completed.

When a control completion notification is received from automaticdriving controller 20, HMI controller 10 generates a schematic diagramincluding a message indicating that control is completed, and outputsthe generated schematic diagram to display device 30. During a periodfor which an operation from a user is not received, HMI controller 10generates a schematic diagram including a message indicating that anoperation is not received, and outputs the generated schematic diagramto display device 30.

FIG. 3 is a diagram illustrating an example of a basic flowchart forexplaining processing of HMI controller 10 in FIG. 1. Determination unit11 of HMI controller 10 checks whether or not a driving mode is anautomatic driving mode or a manual driving mode (S1). In the manualdriving mode (N in S2), the process ends. In a case of the automaticdriving mode (Y in S2), processing is performed as follows.

Sensor information that is input from detection unit 40 to automaticdriving controller 20 is updated at any time (S3). Image generation unit12 of HMI controller 10 generates a schematic diagram including the ownvehicle and the own vehicle peripheral situation based on the ownvehicle position information, the own vehicle traveling roadinformation, and the own vehicle peripheral information that are inputfrom automatic driving controller 20, and draws the generated schematicdiagram on display unit 31 (S4). Determination unit 11 checks whether ornot a reception mode is a reception enable mode in which an operationfrom a user can be received or a reception disable mode in which anoperation from a user cannot be received (S5). In a case where thereception mode is the reception disable mode (N in S6), the processends. In a case where the reception mode is the reception enable mode (Yin S6), determination unit 11 determines whether or not there is acontact to input unit 32 by a user (S7). In a case where there is nocontact to input unit 32 by a user (N in S8), predetermined-time elapsedetermination processing (S12) to be described later is performed. In acase where there is a contact to input unit 32 by a user (Y in S8),determination unit 11 determines a control command according to agesture operation input by a user (S9). The details of the controlcommand determination processing will be described later.

In a case where the control command determined in step S9 is not acancel command (N in S10), image generation unit 12 displays that thecommand is being instructed on display unit 31 (S11). When apredetermined time has elapsed after the control command is determined(Y in S12), in a case where there is a control command determined instep S9 (Y in S13), operation reception disable is displayed on displayunit 31 (S14), determination unit 11 updates the reception mode from thereception enable mode to the reception disable mode (S15), andinstruction unit 13 outputs the determined control command to automaticdriving controller 20 (S16). Until a predetermined time has elapsed (Nin S12), the process transitions to step S3.

In step S10, in a case where the determined control command is a cancelcommand (Y in S10), cancel is displayed (S110), and the process ends. Instep S13, in a case where there is no control command determined in stepS9 (N in S13), an input error is displayed (S111), and the process ends.Automatic driving controller 20 periodically detects the own vehicleposition information, the own vehicle traveling road information, andthe own vehicle peripheral information from detection unit 40. Since theown vehicle peripheral situation constantly changes, after the controlcommand is output to automatic driving controller 20, there is a casewhere it is determined that the control command is not executable. Forexample, after a following instruction, there is a case where other carsinterrupt between the own vehicle and an another vehicle. In a casewhere it is determined that the control command is executable byautomatic driving controller 20 (Y in S17), image generation unit 12displays that the control is being performed on display unit 31 (S18),and starts counting by activating a timer (S19). In a case where it isdetermined that the control is not executable by automatic drivingcontroller 20 (N in S17), image generation unit 12 displays a controldisable error on display unit 31 (S112).

FIG. 4 is a flowchart for explaining update processing of a receptionmode. When a count value of the timer reaches a set value (for example,10 seconds) (Y in S113), determination unit 11 of HMI controller 10updates the reception mode from the reception disable mode to thereception enable mode (S114). The count value of the timer may bechanged according to the own vehicle peripheral situation. When anotification indicating completion of the control is received fromautomatic driving controller 20, or when it is determined that thecontrol according to the control command is completed based on behaviorof vehicle 1, determination unit 11 may update the reception mode fromthe reception disable mode to the reception enable mode.

Hereinafter, in this embodiment, as a control command, an example inwhich a control command for instructing a specific operation using aroad guide sign is issued will be described. A user inputs a gestureoperation for instructing a specific operation using a road guide signto input unit 32. A specific example of the gesture operation will bedescribed later.

FIG. 5 is a flowchart illustrating an example of determinationprocessing in a case where a gesture operation for instructing aspecific operation using a road guide sign is input in step S9 of FIG.3. Determination unit 11 of HMI controller 10 determines whether or notan own vehicle mark is present at a contact start position (S9 a). In acase where the own vehicle mark is not present at the contact startposition (N in S9 a), determination unit 11 determines that the gestureoperation is a control command of another operation in which a roadguide sign is not used (S9 b). In a case where the own vehicle mark ispresent at the contact start position (Y in S9 a), generation unit 12draws a drop enable area in the schematic diagram, and displays the areaon display unit 31 (S9 c). At this time, generation unit 12 generates adrop enable area of a road guide sign in the schematic diagram.Information on the drop enable area of the road guide sign is acquiredfrom a local map information database that is prepared in advance or amap information database on a network. A coordinate of an object such asa place name, an arrow, or the like in the sign may be acquired by usingimage recognition technology based on a road guide sign image capturedin a real time by a vehicle-mounted camera.

Determination unit 11 receives a touch event generated in input unit 32(S9 d), and determines the type of the touch event (S9 e). In a casewhere the type of the touch event is a movement (movement in S9 e),generation unit 12 draws a prediction trajectory/route candidate ofvehicle 1 in the schematic diagram, and the prediction trajectory/routecandidate of vehicle 1 is displayed on display unit 31 (S9 f).

In a case where the type of the touch event is a contact end (contactend in S9 e), determination unit 11 determines whether or not a contactend position is present in the area of the road guide sign (S9 g). In acase where the contact end position is not present in the area of theroad guide sign (N in S9 g), determination unit 11 determines that thegesture operation is a control command of another operation in which theroad guide sign is not used (S9 h). In a case where the contact endposition is present in the area of the road guide sign (Y in S9 g),determination unit 11 determines whether or not the area of the roadguide sign is an area of a lane guide sign or an uphill lane sign (S9i). In a case where the area of the road guide sign is an area of a laneguide sign or an uphill lane sign (Y in S9 i), determination unit 11determines whether or not the lane of a lane guide sign or an uphilllane sign is a right-turn-only lane or a left-turn-only lane (S9 j). Ina case where the lane is not a right-turn-only lane or a left-turn-onlylane (N in S9 j), determination unit 11 determines that the gestureoperation is a traffic lane change instruction command for instructingthe traffic lane change to the lane (S9 k). In a case where the lane isa right-turn-only lane or a left-turn-only lane (Y in S9 j),determination unit 11 determines that the gesture operation is a trafficlane change instruction command for instructing the traffic lane changeto the lane, and a right turn instruction or a left turn instruction (S9l).

In step S9 i, in a case where the dropped area is not an area of a laneguide sign or an uphill lane sign (N in S9 i), determination unit 11determines whether or not the dropped area is an area in which an arrowindicating a direction or a way is displayed (S9 m). In a case where thedropped area is an area in which an arrow indicating a direction or away is displayed (Y in S9 m), determination unit 11 determines that thegesture operation is a traveling direction change instruction commandfor instructing the traveling direction change to the direction (S9 n).In a case where the dropped area is not an area in which an arrowindicating a direction or a way is displayed (N in S9 m), determinationunit 11 determines whether or not the dropped area is an area in which aplace name indicating a city or a region is displayed (S9 o). In a casewhere the dropped area is an area in which a place name indicating acity or a region is displayed (Y in S9 o), determination unit 11acquires a traveling direction corresponding to the place name from themap information database (S9 p). Determination unit 11 determines thatthe gesture operation is a traveling direction change instructioncommand for instructing the traveling direction change to the acquiredtraveling direction (S9 q).

In step S9 o, in a case where the dropped area is not an area in which aplace name indicating a city or a region is displayed (N in S9 o),determination unit 11 determines whether or not the dropped area is anarea in which a road name is displayed (S9 r). In a case where thedropped area is an area in which a road name is displayed (Y in S9 r),generation unit 12 generates an image (for example, a menu or a dialog)or a voice that requests a selection of an inbound and an outbound, andnotifies a user of the image or voice (S9 s). In a case where an inboundis selected by a user (Y in S9 t), determination unit 11 determines thatthe gesture operation is a traveling direction change instructioncommand for instructing the traveling direction change to the inbound(S9 u). In a case where an outbound is selected by a user (N in S9 t),determination unit 11 determines that the gesture operation is atraveling direction change instruction command for instructing thetraveling direction change to the outbound (S9 v).

In step S9 r, in a case where the dropped area is not an area in which aroad name is displayed (N in S9 r), determination unit 11 determineswhether or not the dropped area is an area in which a sign of anentrance, an exit, a parking area, or a service area of a toll road isdisplayed (S9 w). In a case where the dropped area is an area in which asign of an entrance, an exit, a parking area, or a service area of atoll road is displayed (Y in S9 w), determination unit 11 determinesthat the gesture operation is a setting instruction command for settingthe entrance, exit, parking area, or service area of the toll road to apassing spot (S9 x).

In a case where the dropped area is not an area in which a sign of anentrance, an exit, a parking area, or a service area of a toll road isdisplayed (N in S9 w), determination unit 11 determines whether or notthe dropped area is an area in which a sign of a parking lot, a shelter,or an emergency phone is displayed (S9 y). In a case where the droppedarea is an area in which a sign of a parking lot, a shelter, or anemergency phone is displayed (Y in S9 y), determination unit 11determines that the gesture operation is a setting instruction commandfor setting the parking lot, shelter, or road shoulder at which anemergency phone is located to a destination (S9 z). In step S9 y, in acase where the dropped area is not a sign of a parking lot, a shelter,or an emergency phone (N in S9 y), generation unit 12 displays an errormessage on display unit 31 (S9 zz).

Hereinafter, a specific example of the gesture operation in which a roadguide sign is used will be described. In the following examples, it isassumed that a touch panel display in which display unit 31 and inputunit 32 are integrated is used.

FIG. 6 is a flowchart illustrating a processing example of issuing acommand according to the content of the road guide sign by a gestureoperation. Determination unit 11 of HMI controller 10 receives a touchevent (DOWN) from the touch panel (S50). The touch event (DOWN) is anevent representing a contact state change on the touch panel from anon-contact state to a contact state by a finger or a pen. Determinationunit 11 determines whether or not a coordinate detected by the touchevent (DOWN) is present in a display area of the own vehicle icon (S51).In a case where the coordinate detected by the touch event (DOWN) ispresent outside the display area of the own vehicle icon (N in S51), itis determined that the gesture operation is not an operation instructioncorresponding to content of a road guide sign, and the process ends.

In a case where the coordinate detected by the touch event (DOWN) ispresent in the display area of the own vehicle icon (Y in S51),determination unit 11 receives a touch event (MOVE) from the touch panel(S52). The touch event (MOVE) is an event representing a change from acontact state on a certain point of the touch panel to a contact stateon another point of the touch panel by a finger or a pen. Then,determination unit 11 receives a touch event (UP) from the touch panel(S53). The touch event (UP) is an event representing a contact statechange on the touch panel from a contact state to a non-contact state bya finger or a pen.

Determination unit 11 determines whether or not a coordinate detected bythe touch event (UP) is present in a display area of a road guide sign(S54). In a case where the coordinate detected by the touch event (UP)is present in the display area of the road guide sign (Y in S54),instruction unit 13 issues a command according to the content of theroad guide sign to automatic driving controller 20 (S55). In a casewhere the coordinate detected by the touch event (UP) is present outsidethe display area of the road guide sign (N in S54), it is determinedthat the gesture operation is not an operation instruction correspondingto content of the road guide sign, and the process ends.

FIG. 7A and FIG. 7B are diagrams illustrating an example of issuing acourse change instruction command or a route change instruction commandusing a destination guide sign at an intersection. In the schematicdiagram illustrated in FIG. 7A, an own vehicle icon V1, a road, and adestination guide sign S1 are displayed. As display forms of the ownvehicle, the road, and the guide sign, various display forms areconsidered. A real photographed image may be used, and a fine CG imageor an animation image may be used. Display of the own vehicle is notlimited to an icon, and the own vehicle may be displayed as a moresimple mark or character, or may be displayed as a real photographedimage. In other words, there is no problem as long as the own vehicle isdisplayed on a screen as an object in any display form.

In a case where a driver wants to change the course of the own vehicleto any one course displayed on a destination guide sign S1 at anintersection, as illustrated in FIG. 7A, a driver drags the own vehicleicon V1, and as illustrated in FIG. 7B, drops the own vehicle icon V1onto a place name “Machida” N1 of a desired destination in thedestination guide sign S1. Accordingly, a course change instructioncommand for changing the course of the own vehicle to a course of thedesired destination is issued. Alternatively, a route change instructioncommand for changing the route of the own vehicle to a route in whichthe desired destination is set as a passing spot or a destination isissued.

FIG. 8A and FIG. 8B are diagrams illustrating an example of issuing acourse change instruction command or a route change instruction commandusing a traffic lane guide sign. In a case where a driver wants tochange the course of the own vehicle to any one course displayed ontraffic lane guide signs S2 a and S2 b, as illustrated in FIG. 8A, adriver drags the own vehicle icon V1, and as illustrated in FIG. 8B,drops the own vehicle icon V1 onto a desired traffic lane guide sign“Sagamihara” S2 b. Accordingly, a course change instruction command forchanging the course of the own vehicle to a course of the desireddestination is issued. Alternatively, a route change instruction commandfor changing the route of the own vehicle to a route in which thedesired destination is set as a passing spot or a destination is issued.

FIG. 9A and FIG. 9B are diagrams illustrating an example of issuing asetting instruction command for setting a passing spot using an exitguide sign of a toll road. In a case where a driver wants to exit a tollroad through the next exit, as illustrated in FIG. 9A, a driver dragsthe own vehicle icon V1, and as illustrated in FIG. 9B, drops the ownvehicle icon V1 onto an exit guide sign “Yokohama Machida” S3.Accordingly, a passing spot setting instruction command for setting theexit to a passing spot is issued. In a case where the route of the ownvehicle is changed by the setting of a passing spot, a route changeinstruction command is issued.

FIG. 10A and FIG. 10B are diagrams illustrating another example ofissuing a setting instruction command for setting a passing spot usingan exit guide sign of a toll road. In a case where a driver wants toexit a toll road through a desired exit, as illustrated in FIG. 10A, adriver drags the own vehicle icon V1, and as illustrated in FIG. 10B,drops the own vehicle icon V1 onto a desired exit guide sign “TomeiKawasaki” S4 c among multiple exit guide signs S4 a, S4 b, and S4 c.Accordingly, a passing spot setting instruction command for setting theexit to a passing spot is issued. In a case where the route of the ownvehicle is changed by the setting of a passing spot, a route changeinstruction command is issued.

FIG. 11A and FIG. 11B are diagrams illustrating an example of issuing adestination setting instruction command for setting a destination usinga parking lot guide sign. In a case where a driver wants to park the ownvehicle in a parking lot, as illustrated in FIG. 11A, a driver drags theown vehicle icon V1, and as illustrated in FIG. 11B, drops the ownvehicle icon V1 onto a parking lot guide sign S5. Accordingly, adestination setting instruction command for setting the parking lot to adestination is issued. When the destination is set, automatic drivingcontroller 20 reroutes the original destination of the own vehicle to anew destination.

FIG. 12A to FIG. 12C are diagrams illustrating another example ofissuing a course change instruction command using a destination guidesign at an intersection. In a case where a driver wants to change thecourse of the own vehicle to any one course displayed on a destinationguide sign S6 at an intersection, as illustrated in FIG. 12A, a driverdrags the own vehicle icon V1, and as illustrated in FIG. 12B, drops theown vehicle icon V1 onto a road name “Gyoko road” R1 on the destinationguide sign S6. When the own vehicle icon V1 is dropped on the road name,as illustrated in FIG. 12C, a destination display window R2 of theinbound (Machida direction) and the outbound (Zama direction) of theroad is displayed. The inbound or outbound of the destination displaywindow R2 is touched by a driver. Accordingly, a course changeinstruction command for changing the course of the own vehicle to thecourse in the touched direction is issued.

FIG. 13A and FIG. 13B are diagrams illustrating an example of issuing atraffic lane change instruction command and a course change instructioncommand using a guide sign on a traffic lane. In a case where a driverwants the own vehicle to turn right, as illustrated in FIG. 13A, adriver drags the own vehicle icon V1, and as illustrated in FIG. 13B,drops the own vehicle icon V1 onto a right-turn-only lane sign M1 on atraffic lane. Accordingly, a traffic lane change instruction command forinstructing the traffic lane change to the right-turn-only lane and aright turn instruction command are issued.

In the above example, during a period for which the own vehicle icon isdragged and dropped, generation unit 12 may hold the original image (ownvehicle icon before movement) in the schematic diagram, and delete theoriginal image when the own vehicle icon is dropped. During the periodfor which the own vehicle icon is dragged and dropped, generation unit12 may draw a trajectory of the drag of the own vehicle icon using adotted line in the schematic diagram. During the period for which theown vehicle icon is dragged and dropped, generation unit 12 may invertthe color of the road, and return the color of the road to the originalcolor thereof when the own vehicle icon is dropped.

During the period for which the own vehicle icon is dragged and dropped,when there is a drop disable area (an opposite traffic lane or the like)or when an operation (traffic lane change or the like) is disabled,generation unit 12 may change (invert, thin, or the like) the color ofthe own vehicle icon. When the own vehicle icon is dropped, the ownvehicle icon may return to the original position thereof, and an errormessage such as “operation disabled” may be displayed. Examples of acase where an operation is disabled includes approach of followingvehicle, a traffic lane change prohibited area, exceeding of a limitingspeed, or the like.

During the period for which the own vehicle icon is dragged and dropped,when an operation is disabled, generation unit 12 may change (invert,thin, or the like) a color of a background such as a road or the like.When an operation is enabled, the color of the background is returned tothe original color thereof. During the period for which the own vehicleicon is dragged and dropped, generation unit 12 changes a color of thedrop disable area. During the period for which the own vehicle icon isdragged and dropped, when there is a drop disable area or when anoperation is disabled, generation unit 12 performs notification using anerror sound or a vibration.

When the drag of the own vehicle icon is started, in a case where thestart of an operation is disabled, generation unit 12 may change thecolor of the own vehicle icon. The drag operation (movement of the ownvehicle icon) may be disabled. An error message such as “operationdisabled” may be displayed. When the drag of the own vehicle icon isstarted, in a case where the start of an operation is disabled,generation unit 12 may change the color of the background such as a roador the like. When an operation is enabled, the color of the backgroundis returned to the original color thereof. When the drag of the ownvehicle icon is started, in a case where the start of an operation isdisabled, generation unit 12 performs notification using an error soundor a vibration in a state where the drag operation (movement of the ownvehicle icon) is disabled.

During the period for which the own vehicle icon is dropped andprocessing according to the control command is completed, generationunit 12 may display a state of the own vehicle as a ghost, and display atrajectory of the own vehicle. During the period for which the ownvehicle icon is dropped and the own vehicle completes processingaccording to the control command, generation unit 12 may change adisplay state of the own vehicle icon (flashing, a color change, a size,a position, or the like). During the period for which the own vehicleicon is dropped and processing according to the control command iscompleted, determination unit 11 may queue a next instruction as anadditional operation (reserve a control to be performed after thecurrent control is completed).

As described above, according to the present embodiment, it is possibleto transfer the content of various operations to automatic drivingcontroller 20 by moving an icon displayed on the touch panel by agesture operation. The gesture operation of the icon is a simpleoperation, and thus a driver is released from a driving operation in therelated art such as turning of steering 51, depressing of acceleratorpedal 53, or the like. For example, it is possible to easily instructthe operation according to the content of the road guide sign, bydisplaying a schematic diagram including a road guide sign and movingthe own vehicle icon onto the road guide sign. A driver can perform aconfirmation of a peripheral situation and an operation instruction onthe touch panel at the same time, and thus the sight line of the driverdoes not move. Accordingly, it is possible to reduce the possibility ofan erroneous operation and realize safer driving. In order to issue thecontrol command, an operation for linking the own vehicle icon and thesign other than the operation may be used. Although a gesture operationcorresponding to a control command is described as a drag-and-dropoperation or the like, a touch-and touch operation may be used. Apredetermined gesture operation is preferably used, but an operationthat is customizable by a driver may be used. Further, in order torecognize the corresponding relationship of a gesture operation and acontrol command, a comment, an icon, or an arrow may be displayed bydisplay unit 31, or a guide display or a voice guidance may be providedby display unit 31.

As above, the present invention has been described based on theembodiments. These embodiments have been presented by way of exampleonly, and it should be understood by those skilled in the art that theembodiments can be modified in various forms by combining the respectiveelements or processing processes, and the modification examples areincluded in the scope of the present invention.

For example, although an example in which HMI controller 10 isimplemented by a dedicated LSI is assumed, functions of HMI controller10 may be realized using a CPU in a portable device such as a smartphone or a tablet that is used as display device 30. In this case, aportable device that is used as display device 30 and automatic drivingcontroller 20 are directly connected to each other. Functions of HMIcontroller 10 may be realized by a CPU in a head unit such as a carnavigation device, a display audio, or the like. A dedicated LSI onwhich HMI controller 10 is mounted may be included in a head unit.

In the above-mentioned embodiment, an example in which a commandaccording to the display content in a sign is issued by dragging anddropping the own vehicle icon onto a sign object indicating a road guidesign is described. In this regard, similarly, a command according to thedisplay content in a sign can be also issued by dragging and droppingthe sign object onto the own vehicle icon.

The embodiments may be specified by the following items.

[Item 1]

A driving support device (10) including;

an image output unit (14 a) that outputs an image including an ownvehicle object representing an own vehicle and a sign objectrepresenting a sign, to a display unit (31);

an operation signal input unit (14 b) that receives an operation signalof a user for moving, in the image displayed on the display unit (31),the own vehicle object to the position of the sign object or for movingthe sign object to the position of the own vehicle object; and

a command output unit (14 c) that outputs a command according to thedisplay content in the sign object when the operation signal isreceived, to an automatic driving control unit (20) that controlsautomatic driving.

Here, “sign” is a sign on road traffic, and may be a guide sign or atraffic sign.

In this case, a user can intuitively and conveniently perform anoperation according to the content of the sign.

[Item 2]

The driving support device (10) according to Item 1, in which thecommand output unit (14 c) outputs a command for setting, as adestination or a passing spot, a place specified by the display contentin the sign object to which the own vehicle object is moved, to theautomatic driving control unit (20).

In this case, a user can intuitively and conveniently perform anoperation for setting a destination or a passing spot.

[Item 3]

The driving support device (10) according to Item 1 or 2, in which thecommand output unit (14 c) outputs a command for instructing the ownvehicle to enter a traffic lane in the sign object to which the ownvehicle object is moved, to the automatic driving control unit (20).

Here, “enter” includes right turn, left turn, U-turn, and traffic lanechange.

In this case, a user can intuitively and conveniently perform anoperation for instructing right turn, left turn, U-turn, and trafficlane change.

[Item 4]

The driving support device (10) according to Item 3, in which when theoperation signal input unit (14 b) receives an operation signal of theuser for moving the own vehicle object in the image displayed on thedisplay unit (31) to a traffic lane or a road name in the sign object,the image output unit (14 a) outputs an image including a selectionobject for selecting an inbound or an outbound of the traffic lane, tothe display unit (31).

In this case, a user can intuitively and conveniently instruct the ownvehicle to travel in an inbound or an outbound when changing the courseof the own vehicle to another road.

[Item 5]

A driving support system (10, 30) including:

a display device (30) that displays an image; and

a driving support device (10) that outputs an image to the displaydevice (30),

in which the driving support device (10) includes:

-   -   an image output unit (14 a) that outputs an image including an        own vehicle object representing an own vehicle and a sign object        representing a sign, to the display device (30);    -   an operation signal input unit (14 b) that receives an operation        of a user for moving, in the image displayed on the display        device (30), the own vehicle object to the position of the sign        object or for moving the sign object to the position of the own        vehicle object; and    -   a command output unit (14 c) that outputs a command according to        the display content in the sign object when the operation is        received, to an automatic driving control unit (20) that        controls automatic driving.

In this case, a user can intuitively and conveniently perform anoperation according to the content of the sign.

[Item 6]

A driving support method including;

a step of outputting an image including an own vehicle objectrepresenting an own vehicle and a sign object representing a road sign,to a display unit (31);

a step of receiving an operation of a user for moving, in the imagedisplayed on the display unit (31), the own vehicle object to theposition of the sign object or for moving the sign object to theposition of the own vehicle object; and

a step of outputting a command according to the display content in thesign object when the operation is received, to an automatic drivingcontrol unit (20) that controls automatic driving.

In this case, a user can intuitively and conveniently perform anoperation according to the content of the sign.

[Item 7]

A driving support program and a recording medium in which the drivingsupport program is recorded, the driving support program causing acomputer to execute;

processing of outputting an image including an own vehicle objectrepresenting an own vehicle and a sign object representing a sign, to adisplay unit (31);

processing of receiving an operation of a user for moving, t in theimage displayed on the display unit (31), the own vehicle object to theposition of the sign object or for moving the sign object to theposition of the own vehicle object; and

processing of outputting a command according to the display content inthe sign object when the operation is received, to an automatic drivingcontrol unit (20) that controls automatic driving.

In this case, a user can intuitively and conveniently perform anoperation according to the content of the sign.

[Item 8]

An automatic drive vehicle (1) including:

an image output unit (14 a) that outputs an image including an ownvehicle object representing an own vehicle and a sign objectrepresenting a sign, to a display unit (31);

an operation signal input unit (14 b) that receives an operation signalof a user for moving, in the image displayed on the display unit (31),the own vehicle object to the position of the sign object or for movingthe sign object to the position of the own vehicle object;

a command output unit (14 c) that outputs a command according to thedisplay content in the sign object when the operation is received; and

an automatic driving control unit (20) that executes the output command.

In this case, a user can intuitively and conveniently perform anoperation according to the content of the sign.

The present invention can be used in a vehicle equipped with automaticdriving mode.

The invention claimed is:
 1. A driving support device comprising: an image output unit, which, in operation, outputs image information for an image to a display unit, the image including a vehicle icon representing a vehicle and a sign icon representing a place; an operation signal input unit, which, in operation, receives an operation signal of a user: dragging, in the image, the vehicle icon to a position of the sign icon, or dragging, in the image, the sign icon to a position of the vehicle icon; and a command output unit, which, in operation, outputs, in response to the operation signal being received by the operation signal input unit, an automatic driving control command to an automatic driving control unit configured to control automatic driving of the vehicle, the automatic driving control command setting a course of the vehicle to the place.
 2. The driving support device of claim 1, wherein the place is a traffic lane, and the automatic driving control command instructs the vehicle to enter the traffic lane.
 3. The driving support device of claim 1, wherein when the operation signal input unit receives an operation signal of the user dragging, in the image, the vehicle icon to a road name of a road in the sign icon, the image output unit outputs image information to the display unit to include a selection icon for selecting an inbound or an outbound direction of the road.
 4. The driving support device of claim 1, wherein the place is a destination, and the automatic driving control command set a course of the vehicle to the destination.
 5. A driving support system comprising: a display device, which, in operation, displays an image; and a driving support device, which, in operation, outputs image information for the image to the display device, wherein the driving support device includes: an image output unit, which, in operation, outputs image information including a vehicle icon representing a vehicle and a sign icon representing a place, to the display device; an operation signal input unit which, in operation, receives an operation signal of a user: dragging, in the image, the vehicle icon to a position of the sign icon, or dragging, in the image, the sign icon to a position of the vehicle icon; and a command output unit, which, in operation, outputs, in response to the operation signal being received by the operation signal input unit, an automatic driving control command to an automatic driving control unit configured to control automatic driving of the vehicle, the automatic driving control command setting a course of the vehicle to the place.
 6. The driving support system of claim 5, wherein the automatic driving control command sets the place as a destination or a passing spot.
 7. The driving support system of claim 5, wherein the place is a traffic lane, and the automatic driving control command instructs the vehicle to enter the traffic lane.
 8. The driving support system of claim 5, wherein when the operation signal input unit receives an operation signal of the user dragging, in the image, the vehicle icon to a road name of a road in the sign icon, the image output unit outputs image information including a selection icon for selecting an inbound or an outbound direction of the road.
 9. A driving support method comprising; outputting image including a vehicle icon representing a vehicle and a sign icon representing a place; receiving an operation signal of a user: dragging, in the image, the vehicle icon to a position of the sign icon, or dragging, in the image, the sign icon to a position of the vehicle icon; and outputting, in response to receiving the operation signal, an automatic driving control command to an automatic driving control unit that controls automatic driving of the vehicle, the automatic driving control command setting a course of the vehicle to the place.
 10. The driving support method of claim 9, wherein the automatic driving control command sets the place as a destination or a passing spot.
 11. The driving support method of claim 9, wherein the place is a traffic lane, and the automatic driving control command instructs the vehicle to enter the traffic lane.
 12. The driving support method of claim 9, further comprising: receiving an operation signal of the user dragging, in the image, the vehicle icon to a road name of a road in the sign icon; and outputting a selection icon in the image for selecting an inbound or an outbound direction of the road. 